Part 3: Transforming learning with AI

Hi. Welcome to session three of the AI 101 for Teachers Professional Learning Series. In this session we are traveling to the Wharton School at the University of Pennsylvania to chat with Dr. Ethan Mollick, a professor who teaches innovation and entrepreneurship, and Dr. Lilach Mollick, who works on interactive pedagogy and AI research. They will help us explore how AI can be combined with pedagogy to enhance student learning. Let's go meet the Mollicks. Hi I’m Ethan Mollick a professor at Wharton who has been working on how we democratize access to education through tools like games and interactive tools and AI and I’m Lilach Mollick I'm director of Pedagogy at Wharton Interactive, and I've been working at the intersection of AI and education, helping to democratize education for everyone through effective, pedagogically sound use of AI. And we have been working together on the future of education for a while, thinking about how to make education more interactive, to work at scale. And with the advent of general AI, we found a powerful new tool that can really help in the classroom but also carries some risks. Today, we'd like to talk a little bit about the classroom use of AI upsides and downsides to give you some examples to work with. But first, we like to start with our three guiding principles. The first is that AI is undetectable. There are AI tools, detection tools, but they are not effective. The second principle is that AI is ubiquitous. It's everywhere. 169 countries have access to Bing Chat and you and your students have access to the most powerful AI available. The third principle is that AI is transformative. It will transform how we live, how we work, and how we teach and learn. Not only is AI not going away, but this is probably the worst AI you're ever going to use. So if this feels disruptive now kind of have bad news, which is that there isn't a reason to suspect that AI development will not continue. And I think people worry a lot about like the far future or who knows how far it is where AI smarter than humans. But even over the next couple of years, even with this fact of this academic year, I would expect AI to continue to improve. Five times. Ten times? We have no idea. But if you're not already thinking about these systems, what they mean for education, what they mean for you, what they mean for your students careers. I think we have to think about it because these systems are not going to disappear. Let me make the very pragmatic case for why you may want to do this. The first part of the pragmatic case is your students are using this anyway. So you have to come up to speed. I don't think everybody wants to be dragged along this technology. No one asked for education to be massively disrupted. But it is. And unfortunately, we've got to figure out a way to get around that. All your homework assignments can be done by AI now, so you have to think about that. And then I think the second thing is a pragmatic argument about how AI can make your life easier as a teacher. If you put the hours down, you get them back later. And then if you've worked out a number of prompts to help make lives easier for teachers. Yes. So one thing you can do prompts like give me a lesson hook prompts like create a lesson plan or create a quiz for me. So starting to work with your material and the different models could get you give you a really good sense of how the AI works, what it's good at, what it what it's not good at. And so and save you time in the end, right? So that's where I would be my push to teachers is A you have to and B you're going to want to. I think the other important thing is just to try it. They're very simple to use. They're very intuitive because they're conversational. You can continue a conversation and it feels fairly natural. And I think the key really is experimentation. See how it works with you, see how it works within your context, within your topic that you teach. Our rule of thumb is you need about 10 hours time with AI to get what it's good at, what its limitations are. So I would actually start by suggesting that this that the teacher throw their own assignments into the AI and see what kind of results they get back. I would think about asking them to ask their students to create an assignment using AI and then critique that assignment, potentially even in class, to see if the students can get a sense of what the gaps and abilities of AI are. I have a little bit of freedom as an instructor because I'm teaching college and MBA students entrepreneurship. So I have a lot of I have points I want them to make, but they also are building things and doing things and absolutely transformed how that works. So my assignments now literally call for students to do this one impossible thing in class. If you can't code, you have to write working programs. If you have never if you can't do design work, you have to create a full graphic design working prototypes that's literally now part of the class. So where it used to be, write it write a little bit of an essay, do a prototype on paper. Now you have to create a full working product. Every assignment that is written has to be critiqued by at least five famous entrepreneurs through history, and they use AI to invoke those. There's a pedagogical reason, too, which is that entrepreneurs tend to be overconfident. So you want feedback from different sources. So to me it is let me teach ten times more than I did. I used to teach an advanced intermedia entrepreneurship course. I can now, in the intermediate or basic course, get all the way past the advanced material and further. So I think we're going to see that shake out more in the future. But some of this is about powering past what we could do before and I think that's exciting as well. Apart from student tutors as assignments, teachers can certainly use AI coaches, AI assistants to help students prepare for discussions, help students outline, help students do research, help students get feedback on assignments, and just help students develop explanations. I think there are myriad of approaches that are pedagogically sound that teachers can assign to students and watch their work and ask for the back and forth interaction to really see that students are paying attention to and focusing on the material. Let's talk a little bit about AI from a teacher's perspective. So because of the ubiquity of AI, you've got some choices to make in terms of your AI policies in your class. So do you want to permit I do want to forbid AI How are you going to enforce these sorts of things? We're going to assume that you want to use AI to some extent, and we'll dive into a little bit of the details here. So as a instructor, you should know a few things. One is there is obviously ongoing ethical debates about AI, and those are complicated debates. There are debates over whether or not the AI's trained on the right kind of data about the biases I might have about the use of AI and the outcomes for student learning. And it's worth acknowledging these sets of things. But this tool is out there and it is worth thinking about how you want to use it. If you decide that that is okay and how you want to communicate that information beyond the initial ethical concerns, there's also concerns about how AI actually works. So the large language models that power today’s AI don't actually have knowledge of the world. They're predicting the next word. They're predicting the right kinds of sentences or information to give out. And as a result, they make stuff up they hallucinate. So there are often errors or mistakes. Now, it's not always clear those errors or mistakes are worse than the errors and mistakes humans would make. But you need to be aware that there's going to be those kind of errors and mistakes. And then finally, you need to think about as an instructor how you're going to be using AI to aid learning, which means being really clear about what you want to accomplish with an AI tool. They can be used for student learning, but AI's many possible uses in the classroom, so do you want to use them to have student’s generate ideas, which I do in my classes and get better project ideas as a result. Do you want them to use them as tutors to explain concepts to them they don't understand? Do you want the students to get feedback from AI by asking for questions about work that they're doing. Do you want to be a writing companion? Do you want it to explain why quiz answers might be right or wrong? And then once you've decided what who's instructor, you decide what you're going to tell your students. AI detectors don't work. They just don't work. You shouldn't use them. And it's worse than them not working because they have a high false positive rate. That means they select things that’s AI written that aren't AI written and that disproportionately falls on people whose English is a second language. This is just not something that we can do. And I think trying to close the barn door here after it's been opened and try and detect AI is not the future for responsibility in classrooms. The other thing to note too, is that students were using shortcuts in the past. It's not that they weren't using Google, it's not that they weren't using, you know, other students essays. This was happening in the past, but this is a major disruption, and I think it does call for a rethinking of how we do essays. So thinking a little bit more about the learning goal for an essay or the learning goal for any assignment, One of the things that we're noticing as we watch teachers do this is they all feel an obligation to talk about AI and dive deep into the ethical implications of AI and so on. I think that's important, but I don't think that needs to be the theme of every class. I don't think every class needs to be a discussion about AI, just like every class that uses computers doesn't need to be a discussion about computers, I think is important to have that conversation. And right now we're all just reacting so it's not clear who's supposed to have that. So I totally get teachers wanting to have AI discussions, but it's even harder to get up to speed, not just on the use of AI, but how it works. It's, you know, standards, its ethical implications. So I think teachers should feel a little bit of okay ness with experimenting with AI without having to make it the subject of class. First is as Ethan mentioned, that AI can fabricate. That means that any output that the AI gives a student may be made up, it may be mistaken, it may be very subtly mistaken. And so students should be responsible for their own work. They should at the very least check sources, check any number, check any facts that the AI gives them and check them with credible sources. The second principle is that the AI is not a person. It's easy to imbue the AI with a personality or to feel like you're talking to a person, but it's not a person and it doesn't know you. The third principle is really to give it a lot of context. The AI doesn't know you. It doesn't know your context or your experience or your expertise. The more context you give it, the more useful it'll be for you. And the fourth principle is that you're in charge. Not only should you evaluate and interrogate its output, but if it's leading you in a conversation that is no longer useful to you, or if it's stuck in a loop, or if you'd like to change the direction of the conversation, you should absolutely feel free to take charge. So when we talk about AI and these generative AI solutions, we tend to talk about large language models. And there's actually only a few large scale general purpose, large language models. There is the models created by OpenAI, which are GPT 3.5 or GPT 4 GPT 3.5 is the free version that you get through through Chat GPT and GPT 4 is either through the paid Chat GPT or through Microsoft Bing in creative mode. And when we talk about specialized apps, almost all of them are using one of these models and providing prompts and other information on top of it. I generally think instructors should get familiar with the models themselves because those are the models that are actually producing the answers and you can manipulate them directly that way and learn how they work. So if you're trying to buy an off the shelf solution, they're almost certainly using one of these existing models and then providing some sort of wrapper or other information on top of it, and it's often cheaper and more effective and gives you more control to use the foundation models yourself. But that's a choice you get to make. So when developing the prompt, we really and for all of our prompts, we really look at the science of learning and try to combine that with the power of the AI. So for instance, a good tutor pushes you for information. It doesn't just hand to you, a good tutor finds out what you know and builds on that prior knowledge. A good tutor will also find out a little bit about you. A good tutor also knows that you need lots and varied kinds of examples and analogies, and a good tutor knows that the way to you show evidence of mastery is by being able to explain something in your own words to someone else and give an example of it, which is exactly these are exactly the steps in the kinds of questions that we use in the tutor prompt. But you'll notice when you look at our prompts that they do things like provide context the AI as life has discussed already, the idea that it asks you who you are, and we tell the AI who it is. It's an instructor with this kind of setting, you'll notice that it also tells it exactly the scientific framework to use this idea of context matters. We provide controls. We ask it to go step by step through sets of questions to ask, sometimes not in these prompts, we provide examples of good output, and then we tested a lot. You can't do prompting without testing, and that's one of the great things about testing your error expertise. It's cheap to do and so you get to experiment a lot and that makes for good prompts. And we should also say we test it not just on one model but on several models. So for instance, these two prompts we just worked with ChatGPT 4 for, they also work with Bing. Bing will react a little bit differently and it will because it's connected to the Internet. It will also look up citations sometimes. So the right citation, sometimes they're not. But that is available. It may or may not work with some of the other models. So you really have to test it, I think, as an instructor before you give it to your students in the context of the topic that you're teaching to see how it works. So this is all very theoretical, but I think important. So let's let's get practical. Let's talk about some examples of what AI can do. And again, this isn't a monolithic thing, a has many possible uses. As we said, it was transformative earlier. So we're going to show you a couple of prompts that we've created and those will be available to you as well to work with and these are just examples of the ways AI classroom use can work. So the first one we want to show you is a prompt that I believe you created that talks about feedback that gives proper feedback. And one of the really interesting things about the AI side of things is a more sophisticated prompt that takes into account some of the principles we were talking about earlier will result in better outcomes. So students will often ask for writing advice from an AI even if you tell them not to do it. But they're in ask for it in a way that's fairly unsophisticated and is going to give them fairly generic sounding work and possibly more mistakes. If you give a more elaborate prompt, you can get more elaborate answers. So in this case, could you explain what this prompt does, the feedback prompt. Yes. So we combine the principles of good feedback, which is feedback that takes into account your prior knowledge or what you already know from the student perspective takes into account who you are, your learning level, what grade you're in, whether you're in college or you're in a professional. And it also takes into account the idea that you want to respond to this feedback. So it is going to be actionable, it's going to be balanced, it's going to tell you what's wrong and what you can improve on and what you're doing well, and it's going to keep working with you. But like any good tutor or coach, it won't actually give you the answer. It'll push you in that direction, ask you to explain, ask you to construct your own knowledge. And so you can see the prompt hopefully on the screen here and as a place to work from. You don't need to take this as an absolute answer. This is something you can play with, but let's see it in action. So let's get started using this prompt. It says that it's a teaching assistant because that the instructions we gave it and it asks us for our grade level and subject we're studying, what should we say? So I think we're studying Macbeth and we're in 12th grade. In 12th grade. Okay, great. Okay. And so we've told the AI this information it’s feeding it into the logic that it's using here. And it's asking us about a specific assignment and it's asking if we have a rubric or other information to work with or what we're hoping to achieve with with as much information as possible. I don't have a huge amount here, so I'll say I have to write an analysis of Macbeth. It is graded based on rating style and depth of content and you'll see what it's going is it's asking ask questions and soliciting information from us, which makes it kind of a good prompt that you might hand a student better than one that is just they're just developing themselves and it's asking about specific instructions and ask us to share the assignment. Here is what I have written so far, and I have asked the I to generate a Macbeth essay. So here we go. I'm just pasting that in and we'll see what it says here. And you'll notice it's it's working on the information. It's saying it's taking time to carefully read through it. That's a bit of illusion. It's obviously not taking extra time, but it's responding in this method and you'll see it's giving a set of strengths and weaknesses. What's great about, again, using a tutor that you've built or a mentor that you built is that it can give you the kind of feedback that's educationally valuable that ties into pedagogy rather than just students asking Make this essay better. An example of a working in your favor as an educator and not necessarily working against you and undermining the point you're making. So you'll notice at the end, by the way, it gives a question that for the students to answer. So how do you plan to revise your analysis? Give me a plan and specific changes are going again. The kind of thing we would do as an instructor in a classroom soliciting changes or differences. So I think you can start to see why a tool like this can be really useful when properly applied. Now, let's also talk about one other potential use for AI, AI as a tutor. What are some of the advantages and disadvantages of that approach? So an advantage of this approach is that you're getting students to actually pay attention to the material. You're getting them to read over the rubric, to read over the purpose of the essay and the audience, and to really think through it. A disadvantage is that you you certainly can ask the AI to do it for you, but if you work with it and if you're given guidelines to work with it, it's one way to get feedback. But you would then have to evaluate something else that a teacher could do is to ask for the interactions and ask for a reflection about the interactions. What about this feedback was good? What about this feedback was not as good? And again, it's a higher order level thinking about your essay and your process. Excellent. And so why don't you do one other example where the AI acts as a direct instructor and we have a prompt for that as well. There are risks associated with asking the AI to be a direct instructor, which is that hallucination risk. It doesn't know your pedagogy, your your, your perspective. But I find in my classrooms that students are increasingly using the AI as a method of learning so they don't raise their hands as much. So when I ask them why, they're like, Well, I'd rather not show my ignorance in class, I could ask the AI to explain like I'm five. So they're already engaged in this behavior. So something like a tutor both does a useful thing of showing you what the future of AI education might look like. Like the way Khan Academy is building AI interactive tutors to work. And it also might be a tool your students can use to achieve more in class, but you should caveat that with the knowledge that AI tutors are not 100% there yet. But let's let's use an example here. So this tutor is again trying to take the right kind of format. It says, Hello there, I'm your AI tutor and I'm excited to work with you today. What do we want to learn about today? Let's learn about opportunity cost. Opportunity cost. A concept from economics. Let's see what happens. Okay, so we're telling the AI or the opportunity cost that saying it is a key concept economics it’s even throwing in a little emoji here, which is cute. Can you ask us about our learning level? What level are we at here? 11th grade. 11th grade. Now, I wouldn't get too tied up on the individual grade. It's not amazing at differentiating a 10th grader from an 11th grader, but this is part of the context in which it's working in. So that pulling from some sort of universal standards here and it says, what do we know about opportunity cost? So we know that it has to do with alternative choices, has to do with alternative choices. That is it. And of course, one of the advantages of AI is this kind of freeform text and interaction is the real power of education and it's something the AI can fake reasonably well. Again, not as well as a real human instructor yet. And you'll notice it's giving us examples and explaining things in different ways, which is a powerful thing that AI can do. It's very good at breaking things down in different ways, but you'll notice that it's they're starting to ask questions. It's asking us to make choices. So one of things we know from the research on tutoring is you can't just declaim things to people. The value of tutoring comes from soliciting information, making connections. And you can see the AI starting to do this and asking us for connections in our own life. The other thing to mention too, about the tutor prompt is that it is not assuming that the student can judge their own learning. Very often you'll see in a tutor prompt that is very simple. Like explain to me like on ten, it'll ask you if you'll understand something instead. here It's not asking you to make a judgment about your own learning, which we know is inherently flawed. Instead, it's soliciting, as Ethan said, soliciting information from you to find out what you know and to help you build on your knowledge. And these sorts of subtle differences are what separates using AI in sort of an expert way in a classroom where we know what we want to have happen from just the naive use people are doing, I think that there is an advantage to taking charge of your students AI usage because they're going to be using it anyway and thinking about directing it either by giving them prompts, having these discussions and it's a really powerful tool that in the future will greatly boost classroom learning and is not a replacement or threat for teachers. It is something that we can use to improve the outputs of our work, improve student learning, make our lives easier while making students lives better and I think that that's a very powerful view of the future. And I hope that you at least embrace and experiment with AI before deciding whether you want to use it or ban it in your classrooms. Wow. There really are so many ways to enhance student learning using AI. With AI technology advancing rapidly, there will be more and more tools available. As with any new tool, educators have a responsibility to ensure they're using age appropriate tools, protecting student privacy and creating spaces for students to critically evaluate the potential pitfalls of the technology they are using. Join us in session 4 Ensuring a Responsible Approach to AI as we explore these topics. Thanks for joining us. See you again In session four. Visit the AI 101 for Teachers website at Code.org/ai101 to sign up for early access and to explore additional resources from Code.org. ETS, ISTE and Khan Academy. Thanks for joining us.